Probabilistic Databases ∗ Dan

Many applications today need to manage large data sets with uncertainties. In this paper we describe the foundations of managing data where the uncertainties are quantified as probabilities. We review the basic definitions of the probabilistic data model and present some fundamental theoretical results for query evaluation on probabilistic databases. 1 The Quest for Probabilistic Databases Commercial databases today are deterministic. Relational databases are rooted in First Order Logic and Finite Model Theory, as initially envisioned by Codd [10], and were originally motivated by applications like banking, payroll, accounting, inventory, all of which require a precise semantics of the data. Subsequently, databases and data management techniques have been extended to handle richer data models, such as Nested Relations [44], Object-relational data [21], temporal data [45], spatial data [36], and semistructured data and XML [46]. All these extensions rely on a deterministic semantics for both data and queries. Today, the database community needs to manage large volumes of data that is imprecise, or uncertain, and that contains an explicit representation of the uncertainty. Uncertain data occurs in large scale data integration [29], integration of life-science databases [39], in information extraction systems [24, 22, 33], in sensor data [8, 23, 19, 20], in activity recognition data [37, 9, 35]. Modeling and managing data where uncertainties are explicitly represented and numerically quantified requires a new paradigm whose foundation are based on probability theory, probabilistic inference [12], probability logic [2], and degrees of belief [6], in addition to Finite Model Theory. This paper presents the basic definitions of the probabilistic data model and some fundamental theoretical results for query evaluation on probabilistic databases. A precursor to probabilistic databases, ∗This research was supported in part by Suciu’s NSF CAREER grant IIS-0092955, NSF grants IIS-0415193, IIS-0627585, IIS-0513877, IIS-0428168, and a gift from Microsoft.